Communication service method and communication apparatus thereof

ABSTRACT

A communication service method wherein a user ID is registered at server. Video data from the user&#39;s terminal is transmitted to the server. The user selects the setting of whether image data is used with that video data; and the user&#39;s selected setting is stored at the server with the user ID. When the stored setting indicates that image data is used, the video data and the image data are transmitted from the server to one or more other terminals.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a communication service method and acommunication service apparatus, they are able to supply additionalservices such as, for example, an advertisement propagandizing serviceof selling goods, an electronic commerce service and the like, in aprivate communication environment between specific users in acommunication network capable of transmitting arbitrary data includingmoving images. The present invention further relates to a communicationterminal apparatus and a communication system, both capable of receivingsuch services.

2. Description of the Related Art

Owing to the progress of various data processing technologies includingcommunication technologies, coding technologies and the like, a portableterminal apparatus capable of processing a great deal of data such asdata of a moving image.

Conventionally, for example, a Japanese Laid-Open Patent Publication No.Hei 10-164137 discloses an information processing apparatus thatsupplies a previously prepared other piece of information, for exampleadvertisement information, in addition to a piece of information that anuser requested when the apparatus picks up the information in aninformation network. Moreover, the above mentioned Patent Publicationalso discloses that the information to be added is image information,audio information or text information. However, although the PatentPublication discloses exchange process of information between an userterminal and a server, the Patent Publication does not disclose atechnique for adding a previously prepared other piece of informationinto an updated piece of information in the exchange process ofinformation between user terminals.

Then, if the use of a communication network in a broader band, which isexpected in the near future, it is considerable that a communicationterminal capable of processing a great deal of data such as data of areal time moving image (hereinafter, it is sometimes called as a liveimage or moving image) comes into wide use actually.

Moreover, as one of the such communication terminals to become popularwidely, a communication terminal in a visual telephone (TV telephone)form that has conventionally been made to be fit for practical use inconformity with various systems, namely a portable communicationterminal apparatus capable of mutually transmitting live image isconsiderable.

By the way, almost all the communication terminal apparatus in such avisual telephone-type have been placed at a position of an apparatusonly for private communication because only the transmission of liveimage between terminal apparatus to be aimed at.

Then, as a result, there has not been examined the connection betweenthe communication terminal apparatus and the environment of theso-called electronic commerce such as the introduction and theadvertisement of selling goods and moreover the actual dealings that arewidely used in communication networks.

However, it is apparent that such a visual telephone-type communicationterminal apparatus capable of transmitting moving image at real time iseffective as a system related to such electronic commerce, and moreeffective services and systems using such communication terminalapparatus have been desired.

SUMMARY OF THE INVENTION

Therefore, an object of the present invention is to provide acommunication service method and an apparatus therefore that can supplyan advertisement propagandizing service of selling goods and anelectronic commerce environment in a private communication environmentamong specific users.

Moreover, another object of the present invention is to provide acommunication terminal apparatus capable of receiving an advertisementpropagandizing service of selling goods and an electronic commerceenvironment while performing private communication with a specific user.

Moreover, a further object of the present invention is to provide acommunication system capable of performing an advertisementpropagandizing service of selling goods and capable of being connectedwith electronic commerce while being in a private communicationenvironment with a specific user.

Moreover, a further object of the present invention is to provide anadvertisement propagandizing method capable of performing anadvertisement propagandizing service of selling goods in a privatecommunication environment among specific users.

For resolving the aforesaid problems, a communication service methodaccording to the present invention connects a plurality of communicationterminal apparatus in a state possible to transmit an audio and an imagesignals, and superposes an image signal concerning arbitrary sellinggoods to an image signal transmitted from an arbitrary one of thecommunication terminal apparatus, and delivers the image signalsuperposed with the image signal of the arbitrary selling goods to thecommunication terminal apparatus.

Preferably, the image signal of selling goods is selected by an user ofthe communication terminal apparatus that originates the image signal tobe superposed with the image signal of the selling goods.

Moreover, preferably, the image signal originated from the arbitrarycommunication terminal apparatus is an image signal including an imageof the user of the communication terminal apparatus.

Moreover, preferably, the image signal transmitted among the pluralcommunication terminal apparatus is an image signal including a realtime moving image.

Moreover, preferably, in response to a demand from the communicationterminal apparatus, to which the image signal is delivered, moredetailed information of the selling goods is supplied to thecommunication terminal apparatus.

Moreover, preferably, an environment for performing transactions of theselling goods is supplied to the communication terminal apparatus towhich the more detailed information of the selling goods is supplied.

Moreover, preferably, the image signal of the selling goods issuperposed with the image signal originated from the communicationterminal apparatus so that the image signal of the selling goods hassubstantially significant relation with an object included in the imagesignal originated from the communication terminal apparatus.

Moreover, preferably, the image signal of the selling goods issuperposed with the image signal originated from the communicationterminal apparatus so that an image in which the user of thecommunication terminal apparatus puts on the selling goods, uses theselling goods or holds the selling goods is generated.

Preferably, a consideration is paid to the user of the communicationterminal apparatus to which the image signal of the selling goods isdelivered in a state of being superposed with the originated imagesignal on a basis of a prescribed condition concerning the delivery ofthe image signal of the selling goods.

Specifically, the amount of the consideration is determined based on anumber of times of the delivery of the superposed image signal of theselling goods, a period of time of the delivery, or a prescribed stateto prescribed processing concerning the selling goods performed by theuser of the communication terminal apparatus of a destination of thedelivery.

Moreover, specifically, the consideration is paid by money, bycountervailing of the consideration with a utility rate of thecommunication service, or by countervailing of the consideration withmoney to be paid at a time of prescribed selling goods transactions.

Specifically, the selling goods the image signal of which is superposedare further selected according to the user of the communication terminalapparatus to which the superposed image signal is delivered.

Preferably, the selling goods the image of which is superposed arechanged in sequence at every series of the connection, at everyprescribed period of time, in response to a demand from thecommunication terminal apparatus that originated the image to besuperposed, or in response to a demand from the communication terminalapparatus that is a destination of the superposed image.

Moreover, a communication service apparatus according to the presentinvention comprises: connecting means for connecting a plurality ofcommunication terminal apparatus in a state possible to transmit a audioand an image signals; receiving means for receiving the image signaloriginated from the connected communication terminal apparatus; imagesignal superposing means for superposing an image signal concerningarbitrary selling goods to the received image signal; and transmittingmeans for transmitting the superposed image signal to another of theconnected communication terminal apparatus.

Preferably, the apparatus further comprises first recording means forrecording the selling goods selected beforehand by each user concerningthe communication terminal apparatus, the selling goods being superposedwith the image signal originated from the communication terminalapparatus concerning the user, wherein the image signal superposingmeans selects the selling goods based on contents recorded in the firstrecording means, and superposes an image signal of the selected sellinggoods.

Moreover, preferably, the apparatus further comprises second recordingmeans for recording information concerning the selling goods to beadvertised, the information including image data of the selling goods,wherein the image signal superposing means selects the selling goodsamong various selling goods recorded in the second recording means, andperforms superposition of the image signal by using the recorded imagedata of the selling goods.

Preferably, the connecting means connects the plural communicationterminal apparatus in a state possible to transmit a real time movingimage signal among the communication terminal apparatus.

Preferably, the image signal originated from the communication terminalapparatus includes an image of an user of the communication terminalapparatus, and the image signal superposing means superposes the imagesignal of the selling goods with the image signal originated from thecommunication terminal apparatus so that the image in which the user ofthe communication terminal apparatus puts on the selling goods, uses theselling goods or holds the selling goods is generated.

Moreover, preferably, the image signal superposing means changes theselling goods in sequence at every series of the connection, at everyprescribed period of time, in response to a demand from thecommunication terminal apparatus that originated the image signal, or inresponse to a demand from the communication terminal apparatus that is adestination of the superposed image signal.

Moreover, preferably, the communication service apparatus furthercomprises: third recording means for recording prescribed dataconcerning actual results of superposition of image data of the sellinggoods with image data originated from the communication terminalapparatus at every user concerning the communication terminal apparatus;and adjusting means for paying a consideration to each user on a basisof contents recorded in the third recording means.

Moreover, preferably, the communication service apparatus furthercomprises fourth means for recording a hobby and a preference of theuser of the communication terminal apparatus that is connected by theconnecting means and receives delivery of the superposed image signal,wherein the image signal superposing means selects the selling goodsfurther on a basis of contents recorded in the fourth recording means,and superposes the image signal of the selected selling goods.

Moreover, the transmitting means further transmits information of adestination of connection of a system supplying more detailedinformation of the selling goods or an environment for performingtransactions of the selling goods to the communication terminalapparatus that transmitted the superposed image signal.

Moreover, a communication terminal apparatus according to the presentinvention is connected with another communication terminal apparatusmutually through a communication service apparatus, the communicationterminal apparatus performing mutual telephone calling with the othercommunication terminal apparatus, the communication terminal apparatuscomprises: telephone calling means for performing telephone calling withthe connected other communication terminal apparatus; receiving meansfor receiving image data of a superposed image of an image signaloriginated from the other communication terminal apparatus and an imagesignal concerning arbitrary selling goods and a prescribed control data,the image data and the control data being transmitted from thecommunication service apparatus; displaying means for displaying thereceived image data; position designating means for designating anarbitrary position on the displayed image data; and controlling meansfor performing prescribed processing on a basis of the received controldata when the designated position is a part of the superposed image ofthe image data.

Specifically, the controlling means demands more information of theselling goods concerning the superposed image signal or an environmentconcerning transactions of the selling goods to the communicationservice apparatus when the part of the superposed image is designated.

Moreover, specifically, the controlling means demands more informationof the selling goods or an environment concerning transactions of theselling goods to the other communication service apparatus concerningthe superposed image when the part of the superposed image isdesignated.

Preferably, the receiving means receives information of a replyingresult to the demand transmitted from the other communication serviceapparatus concerning the selling goods, and the displaying meansdisplays the information of the received replying result.

Moreover, preferably, processing concerning the information from theother communication service apparatus in the receiving means and thedisplaying means and processing concerning the other communicationterminal apparatus connected through the communication service apparatusare performed at a substantially same time.

Moreover, a communication system according to the present inventionincludes a plurality of communication terminal apparatus and acommunication service apparatus for connecting the plural communicationterminal apparatus possible to transmit an audio and an image signals,wherein the communication service apparatus comprises: receiving meansfor receiving an image signal originated from one of the connectedcommunication terminal apparatus; image signal superposing means forsuperposing an image signal concerning arbitrary selling goods with thereceived image signal; and transmitting means for transmitting thesuperposed image signal to another of the connected communicationterminal apparatus, and the communication terminal apparatus comprises:receiving means for receiving image data originated by another of thecommunication terminal apparatus and transmitted from the communicationservice apparatus; and displaying means for displaying the receivedimage data.

Moreover, an advertisement propagandizing method according to thepresent invention, in a communication environment in which a pluralityof communication terminal apparatus are connected in a state that anaudio and an image signals can be transmitted, the advertisementpropagandizing method selects selling goods of a desired advertisementpropaganda object by an user of the communication terminal apparatus,and superposes an image signal of the selected selling goods of theadvertisement propaganda object with an image signal originated by thecommunication terminal apparatus concerning the user, and delivers theimage signal in which the image signal of the selling goods issuperposed to another communication terminal apparatus.

Moreover, the image signal originated from the communication terminalapparatus includes an image signal of the user of the communicationterminal apparatus.

Moreover, the image signal of the selling goods is superposed with theimage signal originated from the communication terminal apparatus sothat an image in which the user of the communication terminal apparatusputs on the selling goods, uses the selling goods or holds the sellinggoods is generated.

Moreover, more detailed information of the selling goods or atransaction environment of the selling goods is supplied to thecommunication terminal apparatus to which the image signal is deliveredin response to a demand from the communication terminal apparatus.

Moreover, a consideration is paid to the user of the communicationterminal apparatus, to which the superposed image signal of theoriginated image signal and the image signal of the selling goods aredelivered, on a basis of a prescribed condition concerning the deliveryof the image signal of the selling goods.

Moreover, the selling goods the image signal of which is superposed arefurther selected according to the user of the communication terminalapparatus to which the superposed image signal is delivered.

According to the present invention, there can be provided acommunication system, a communication service method and the apparatusthereof that are possible to provide an advertisement propagandizingservice of selling goods and an electronic commerce environment in aprivate communication environment between specific users.

Moreover, there can be provided a communication terminal apparatus thatcan receive an advertisement propaganda service of selling goods and anelectronic commerce environment while performing a private communicationwith a specific user.

Moreover, there can be provided an advertisement propagandizing methodthat can perform an advertisement propaganda service of selling goods ina private communication environment among specific users.

Moreover, there can be provided an information processing apparatus thatsuperposes the advertisement propaganda information of desired sellinggoods with transmission data.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentinvention will become more apparent from the following description ofthe presently preferred exemplary embodiments of the invention taken inconjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram showing the configuration of a communicationsystem of an embodiment of the present invention;

FIG. 2 is a block diagram showing the configuration of an user terminalapparatus of the communication system shown in FIG. 1;

FIG. 3 is a block diagram showing a configuration of a service businessenterprise system of the communication system shown in FIG. 1;

FIG. 4 is a block diagram showing the configuration of a part in theadvertisement information adding section of the service businessenterprise system shown in FIG. 3 for processing the addition ofadvertisement image data to image data;

FIG. 5 is a block diagram showing the configuration of the image signalsuperposing area extracting section in the advertisement informationadding section shown in FIG. 4;

FIG. 6 is a figure for illustrating the operations of the communicationsystem shown in FIG. 1;

FIG. 7 is a figure showing a live image signal to be transmitted to aservice business enterprise system from an user terminal apparatus onthe transmission side in the communication system shown in FIG. 1;

FIG. 8 is a figure showing an image signal, in which image signal ofselling goods is superposed, to be transmitted to the user terminalapparatus on the transmission side from the service business enterprisesystem in the communication system shown in FIG. 1;

FIG. 9 is a first figure showing another form to superpose an imagesignal of selling goods in the service business enterprise system shownin FIG. 1;

FIG. 10 is a second figure showing a further form to superpose an imagesignal of selling goods in the service business enterprise system shownin FIG. 1;

FIG. 11 is a third figure showing a further form to superpose an imagesignal of selling goods in the service business enterprise system shownin FIG. 1;

FIG. 12 is a fourth figure showing a further form to superpose an imagesignal of selling goods in the service business enterprise system shownin FIG. 1;

FIG. 13 is a fifth figure showing a further form to superpose an imagesignal of selling goods in the service business enterprise system shownin FIG. 1;

FIGS. 14A-14D are figures showing the displayed states of introductionscreens of selling goods on the display of the user terminal apparatusshown in FIG. 2;

FIG. 15 is a block diagram showing the configuration of the part forprocessing the addition of advertisement data to audio data in theadvertisement information adding section of the service businessenterprise system shown in FIG. 3;

FIG. 16 is a block diagram showing an example of another configurationof the service business enterprise system shown in FIG. 3; and

FIG. 17 is a block diagram showing an example of another configurationof the communication system shown in FIG. 1.

DESCRIPTION OF THE PREFERRED EMBODIMENT

An embodiment of the present invention is described by reference toattached drawings.

In the present embodiment, the present invention is described by theillustration of a communication system in a visual (TV or video)telephone-form that is capable of transmitting live image signalsbetween callers mutually, especially, capable of supplying a newcommercial transaction environment by utilizing ordinary privatecommunication as advertisement means.

First, the configuration of the communication system is described byreference to FIGS. 1-5.

FIG. 1 is a block diagram showing a basic configuration of acommunication system 11.

As shown in the FIG. 1, the communication system 11 has a plurality ofuser terminal apparatus 100 connection system 200, a service businessenterprise system 300 and a plurality of sponsor systems 400 as itsbasic configuration.

The user terminal apparatus 100 are terminal apparatus for performingconversations between users by means of audio and image signals, and atthe same time are terminal apparatus for performing the setting or theoperations of communication services provided by the service businessenterprise system 300 and for receiving selling goods transactionservices provided by the sponsor systems 400.

The user terminal apparatus 100 have functions such as audio telephonecalls, the display of images and the taking of images.

The user terminal apparatus 100 are described by reference to FIG. 2.

The user terminal apparatus 100 includes a video camera 102, an imagecoding section 104, an image decoding section 106, a display section108, a microphone 110, a voice coding section 112, a voice decodingsection 114, a speaker 116, an input section 118, a memory 120, acommunication I/F (Inter-Face) 122 and a CPU (Central Processing Unit)124.

The camera 102 is a constituent element for taking image of an user totransmit a live image signal to a communication companion. Namely, thecamera 102 is a small-sized CCD (Charge Coupled Device) camera disposedat a position where at least an image of a face of the user can be takenin a state that the user is ordinarily using the user terminal apparatus100. Image signal signals obtained by the taking with the camera 102 areoutput to the image coding section 104.

The image coding section 104 codes the image signals input from thecamera 102 in conformity with a prescribed system to output the codedsignals to the communication I/F 122.

Coded image data of a communication companion, or coded image data towhich advertisement information is added by the service businessenterprise system 300, which will be described later, and coded imagedata of selling goods transmitted from the sponsor system 400, whichwill be described later, in some cases, are input into the imagedecoding section 106 from the communication I/F 122. The image decodingsection 106 decodes the input coded image data to output the decodedimage data to the display section 108. There are a case where onlyeither of the two image data is input and a case where both of them areinput substantially at the same time. Even in the case where both ofthem are simultaneously input, the image decoding section 106 decodesboth of them substantially at the same time, so that it is possible todisplay them on the display section 108 at the same time, and outputsthe decoded data to the display section 108.

The display section 108 displays the image data input from the imagedecoding section 106, for example, text data concerning selling goodsinput from the communication I/F 122 directly from the sponsor system400, which will be described later, the information related to the usagestates and the operations of the user terminal apparatus 100 input fromthe CPU 124, and the like on a display so that an user can visuallyrecognize them.

Although it is not shown, the display section 108 has a display thatdisplays images actually and a control section that generates signals tobe superposed on the display. The image data input by the image decodingsection 106 from the communication companion, the image data of theselling goods, the text data input from the communication I/F 122 andthe CPU 124, and the like are superposed to a signal to be displayed ona screen by a control section, and are converted into a signal suitablefor being input on the display, for example, and they are converted intoan analog signal. Then the converted analog signal is imposed on thedisplay to be displayed.

The microphone 110 takes in the outside voices or sound including anuser's voice at a time of telephone calling, and generates an audiosignal to output the generated signal to the voice coding section 112.

The voice coding section 112 codes the audio signal input from themicrophone 110 in conformity with a prescribed system to output thecoded signal to the communication I/F 122.

The voice decoding section 114 decodes the coded audio data that isreceived by the communication I/F 122 and is input into the voicedecoding section 114, and converts the coded audio data into an analogsignal capable of being output from the speaker 116 to impose theconverted signal on the speaker 116.

The speaker 116 outputs the audio signal input from the voice decodingsection 114 as a voice.

The input section 118 is an operation section for performing variousoperations of an user such as the dialing to a destination of telephonecalling, the setting of various functions of the user terminal apparatus100, the setting of various services to the service business enterprisesystem 300, the designation of selling goods included in a live imagesignal from a communication companion, the operation for picking up theinformation, the various operations for selling goods introductionscreen transmitted from the sponsor system 400. The input section 118has a ten-key and an arbitrary pointing device.

The memory 120 memorizes a computer software program for the processingof the control of the whole user terminal apparatus 100 performed by theCPU 124 and various data such as setting data of the functions and theoperations of the user terminal apparatus 100, data during processing inthe CPU 124, data being transmitted between each constituent section,and data during processing in each constituent section.

Moreover, as the data related especially to the present invention, thedestination information of the connection of the sponsor system 400 ofthe information of selling goods included in a live image signal from acommunication companion for obtaining the more detailed information ofthe selling goods is memorized in the memory 120 in sequence.

Incidentally, in the present embodiment, when the information of aplurality of selling goods is received at one time of communication, allof the destination information of connections of the sponsor systems 400of the selling goods is memorized in the memory 120. Moreover, thestored information is erased by the user terminal apparatus 100 when thenext communication begins.

The communication I/F 122 performs the modulation and the demodulationof data, protocol control and so forth, and communicates with theoutside actually.

Voice data input from the microphone 110 and coded by the voice codingsection 112 and image data taken by the camera 102 and coded by theimage coding section 104 are transmitted from the communication I/F 122.Incidentally, the transmitted data are finally transmitted to acommunication companion through the connection system 200 and theservice business enterprise system 300.

Moreover, the data transmitted from a communication companion throughthe service business enterprise system 300 and the connection system 200is received by the communication I/F 122, and then the image datathereof is output to the image decoding section 106, and the voice datathereof is output to the voice decoding section 114, and the text datathereof is output to the display section 108, and the control datathereof is output to the CPU 124, respectively.

Moreover, when an user wants to obtain more detailed information ofselling goods on the basis of the information of the selling goodsincluded in a live image from a communication companion, thecommunication I/F 122 transmits a demand for the more detailedinformation of the selling goods generated by the CPU 124 on the basisof an operation of the input section 118 by the user to the sponsorsystem 400 concerning the selling goods through the connection system200.

Moreover, when some information is transmitted from the sponsor system400 in response to the demand, or when the connection with the sponsorsystem 400 is established, the communication I/F 122 receives theinformation transmitted from the sponsor system 400, and the image datathereof is output to the image decoding section 106, and the voice datathereof is output to the voice decoding section 114, and the text datathereof is output to the display section 108, and the control datathereof is output to the CPU 124, respectively.

The CPU 124 operates in conformity with programs memorized in the memory120 (preferably a ROM (Read Only Memory)) and controls each section ofthe user terminal apparatus 100 to perform in desired operations.

As the processing concerning the present invention, for example, thereis the processing of the communication I/F 122 at the time of receivingthe information of the advertisement information of selling goods addedto a live image from a communication companion. In the processing, thecommunication I/F 122 extracts the destination information of connectionof the sponsor system 400 concerning the selling goods included in theadvertisement information, and memorizes the extracted destinationinformation of connection in the memory 120 (preferably a RAM (RandomAccess Memory in this case)).

Then, when the operation for obtaining the detailed information of theselling goods is performed by the user, the CPU 124 requires thedetailed information to the sponsor system 400 concerning the sellinggoods through the communication I/F 122 on the basis of the destinationinformation of connection stored in the memory 120.

Moreover, when the CPU 124 performs the actually processing concerning acommercial transaction as a result of the communication with the sponsorsystem 400, the CPU 124 performs an authentication processing, anadjustment processing and the like therefor.

Moreover, when communication with a new destination of communication isbegun, the CPU 124 performs the processing of erasing the destinationinformation of the connection, which is stored in the memory 120, of thesponsor system 400 of the selling goods the advertisement information ofwhich is received in the communication at the last time.

In the user terminal apparatus 100 having such a configuration, thecommunication I/F 122 begins communication with an outside apparatus inconformity with the control of the CPU 124 by, for example, an operationof the input section 118 by an user.

After the securing of a channel, a voice pronounced by the user is takenin by the microphone 110 to be converted into a digital coded signal bythe voice coding section 112. An image of the user is taken by thecamera 102 to be converted into a digital coded signal by the imagecoding section 104. Moreover, additional information, controlinformation and the like are generated by the CPU 124 on the basis of,for example, an operation of the input section 118 by the user as theneed arises, and the generated information is transmitted through thecommunication I/F 122 respectively.

Moreover, audio signals and image signals in the form of digital codedsignals, and additional information and control information as the needarises, are transmitted also from the outside to the user terminalapparatus 100. In the user terminal apparatus 100, the communication I/F122 receives these signals and information, and the audio signals aredecoded by the voice decoding section 114 to be output by the speaker116, and the image signals are decoded by the image decoding section 106to be displayed by the display section 108.

Moreover, the additional information and the control information areappropriately processed by the CPU 124 on the basis of the information.

The communication in the visual telephone-form by the use of live imageswith an arbitrary companion becomes possible by such an operation of theuser terminal apparatus 100.

Moreover, in the present embodiment, signals composed of image signals,audio signals, both being transmitted from a communication companion,and some advertisement information of selling goods added to the imagesignals and the audio signals are transmitted to the user terminalapparatus 100 by the service business enterprise system 300, which willbe described later. The advertisement information to be added includesimage data of the selling goods to be added to the image signals, theinformation to be transmitted as control signals for prescribing theadded image data part, the connection information of the sponsor system400 concerning the selling goods, the information for prescribing theoperations of the user terminal apparatus 100 in the case where theimage data is selected by the user, and the like. When such signals aretransmitted, the image signals thereof and the audio signals thereof aredecoded by the image decoding section 106 and the voice decoding section114, respectively, and are output by the display section 108 and thespeaker 116, respectively, as described above.

The control signal is once memorized in the memory 120 by the CPU 124,and is referred in the case where the user performs some operation ofthe input section 118, or the like. That is, prescribed processing isexecuted on the basis of the stored information that prescribesoperations. To put it more concretely, for example, the followingprocessing is performed: the operations and instructions of the user aredetected by reference to the information prescribing a stored image datapart; the detailed information concerning selling goods is demanded tothe sponsor system 400 by reference to stored connection information;and the connection with the sponsor system 400 is performed.

The user terminal apparatus 100 performs the communication with thesponsor systems 400 after that, and the processing such as theacquisition of the information concerning the selling goods or the orderof the selling goods is executed by the aforesaid communicationfunction.

However, in the case where the user terminal apparatus 100 has anauthentication function or the function of specific adjustmentprocessing, the CPU 124 utilizes these functions when actual sellinggoods transactions are performed.

Moreover, as mentioned above, the communication I/F 122 can performsimultaneous communication with a plurality of destinations ofcommunication, and also the display section 108 can display a pluralityof images at the same time. Consequently, it is possible to do at thesame tune the communication with a desired companion in the visualtelephone-form, which is described first, and the communication with thesponsor system 400 concerning the selling goods that is detected at thattime and the display of the information of the selling goods.

The mentioned above are the configuration of the user terminal apparatus100 and the operations thereof.

The connection system 200 provides a communication network comprisingnodes including the user terminal apparatus 100, the service businessenterprise system 300 and the sponsor systems 400, and the connectionsystem 200 connects arbitrary nodes to each other in the communicationnetwork in response to a demand from each of the node apparatus toenable communication between each apparatus.

As the processing related to the present embodiment, because the userterminal apparatus 100 makes a connection demand on the service businessenterprise system 300 primarily in the case where communication betweenthe user terminal apparatus 100 is performed, the connection system 200connects the user terminal apparatus 100 with the service businessenterprise system 300 in the network on the basis of the connectiondemand. Then, the connection system 200 secondarily connects the servicebusiness enterprise system 300 with the user terminal apparatus 100 thatis a telephone call companion in response to an demand of the servicebusiness enterprise system 300 after that.

Thereby, there is provided an environment in which the user terminalapparatus 100, which demanded the connection, and the user terminalapparatus 100 being a communication companion can communicate throughthe service business enterprise system 300.

Moreover, the connection system 200 connects the user terminal apparatus100 and the sponsor system 400 in the network on the basis of a demandfrom the user terminal apparatus 100 or the sponsor system 400.

Moreover, when the service business enterprise system 300 introduces anuser to the sponsor system 400 or the like, the connection system 200transmits the information concerning the user from the service businessenterprise system 300 to the sponsor system 400 or connects them witheach other in response to a demand from the service business enterprisesystem 300.

Incidentally, the connection system 200 connects each apparatus bychannels in various forms, but the connection system 200 connects theuser terminal apparatus 100 to each other by means of a channel havingan enough bandwidth through which real-time moving image signals can betransmitted finally.

The service business enterprise system 300 is a system that provides acommunication environment in which advertisement information intervenesin usual private communication to each of the user terminal apparatus100.

The service business enterprise system 300 basically mediates theconnection among a plurality of the user terminal apparatus 100 andprovides communication services in the visual telephone-form thatincludes the mutual transmission of moving image signals to these userterminal apparatus 100 and, at that time, the service businessenterprise system 300 makes the addition of advertisement informationand the development to selling goods transaction processing therefrompossible by superposing the image data of desired selling goods selectedby an user himself or herself with a transmitted live image signal, andfurther by adding information enabling the immediate communication withthe sponsor system 400 concerning the selling goods in response to ademand of an user on the reception side.

The service business enterprise system 300 having the aforesaidfunctions will be described by reference to FIGS. 3-5.

FIG. 3 is a block diagram showing the constitution of the servicebusiness enterprise system 300.

As shown in FIG. 3, the service business enterprise system 300 comprisesa communication I/F 310, advertisement information adding section 320, adatabase I/F 330, an user database 350, a sponsor database 360, acontract-related database 370, an advertisement information database 380and a control section 340.

The communication I/F 310 is an interface that communicates with anarbitrary destination of communication through the connection system200, and communicates especially with the user terminal apparatus 100 inthe present embodiment.

When the user terminal apparatus 100 makes a connection demand on thecommunication I/F 310 through the connection system 200, thecommunication I/F 310 receives the demand and establishes a channel withthe user terminal apparatus 100. Then, when a signal indicating somedemand, setting and so forth is input from the user terminal apparatus100, the communication I/F 310 outputs the signal to the control section340.

In the case where the user terminal apparatus 100 demands the setting ofa communication environment and a communication condition, or the likesuch as the alteration of the condition concerning the processing of theaddition of advertisement information, the selection of selling goods ora sponsor, or the like, because some response to the demand is inputinto the communication I/F 310 from the control section 340, thecommunication I/F 310 transmits the response back to the user terminalapparatus 100.

Moreover, when the user terminal apparatus 100 demands the communicationwith another user terminal apparatus 100, because the communication I/F310 is instructed by the control section 340 to connect the userterminal apparatus 100 with the user terminal apparatus 100 of thecommunication companion on a line, the communication I/F 310 demands theconnection to the connection system 200 on the basis of the instruction.Thereby, a channel is established between the user terminal apparatus100 that demanded the communication and the user terminal apparatus 100of the communication companion.

After the channel is established, the communication I/F 310 in receivessignals including image signals and audio signals that are input fromeach user terminal apparatus 100, and outputs them to the advertisementinformation adding section 320. Moreover, the communication I/F 310transmits signals including image signals and audio signals that areinput from the advertisement information adding section 320 to each userterminal apparatus 100.

The advertisement information adding section 320 adds advertisementinformation to the transmission data between the user terminal apparatus100, which are input from the communication I/F 310, as the occasiondemands in conformity with the setting by the user who concerns the userterminal apparatus 100 that transmitted the transmission data.

Audio data, image data and control data that are transmitted from eachuser terminal apparatus 100 are input to the advertisement informationadding section 320 through the communication I/F 310.

Because the advertisement information adding section 320 does not addany advertisement information to the audio data in the presentembodiment, the advertisement information adding section 320 outputsinput audio data to the communication I/F 310 as they are as thetransmission data to the user terminal, apparatus 100 of thecommunication companion.

As for the image data, in the case where an user who transmitted thetransmission data set to add advertisement information, theadvertisement information adding section 320 superposes the image dataof the advertisement information of specific selling goods selected bythe user that is input from the database I/F 330 into a part of thetransmitted image data that is, for example, a live image of the userand outputs the superposed image data to the communication I/F 310 astransmission data to the user terminal apparatus 100 of the telephonecalling companion.

In particular, the advertisement information adding section 320 of thepresent embodiment superposes the image data of selling goods of theobject of advertisement in a way that is fitted to the live image of anuser. To put it more concretely, the advertisement information addingsection 320 superposes the image signal in the form that the imageindicating the selling goods is used by the user or used as thebackground thereof, or in the form that the selling goods itself is wornby the user or held by the user or laid in the space in which the userexists.

In the case where the advertisement image data are thus superposed, inorder that the user can actually obtain further information concerningthe selling goods or can perform the transaction thereof on the basis ofthe advertisement information, the advertisement information addingsection 320 adds additional information of the advertisement informationof the selling goods that is input from the database I/F 330 togetherwith the image data, or, to put it concretely, the informationprescribing a superposed image data part, the connection information ofthe sponsor system 400 concerning the selling goods, and the informationprescribing the operations of the user terminal apparatus 100 when theimage data is selected by the user related to the user terminalapparatus 100, to the control data transmitted from the user terminalapparatus 100, and the advertisement information adding section 320outputs the added control data to the communication I/F 310 astransmission data to the user terminal apparatus 100 of the telephonecalling companion.

Moreover, in the case where an user did not set to add the advertisementinformation, both of the image data and the control data, which areinput from the communication I/F 310 into the advertisement informationadding section 320, are output to the communication I/F 310 from theadvertisement information adding section 320 as they are as thetransmission data to the user terminal apparatus 100 of thecommunication companion like the audio data.

Incidentally, the processing of investigating the setting by an user andof controlling the operations of the advertisement information addingsection 320 when the advertisement information adding section 320performs such processing is actually done by the control section 340that will be described later. Accordingly, the advertisement informationadding section 320 performs the aforesaid processing by the use of theadvertisement image data that is input from the database I/F 330 by thecontrol of the control section 340 as the need arises on the basis ofcontrol signals that are input from the control section 340.

As for the constitution of the advertisement information adding section320 for performing such processing, especially, the constitution of apart thereof for processing the addition of advertisement image data toimage data will be described by reference to FIGS. 4 and 5 moreminutely.

FIG. 4 is a block diagram showing the constitution of a part thatperforms the processing to add advertisement image data to the imagedata in the advertisement information adding section 320.

The advertisement information adding section 320 has an image signalsuperposing area extraction section 321, a preprocessing section 326, animage superposing section 327 and a switching section 328.

The image superposing area extracting section 321 decides an object theadvertisement image data of which is superposed on the basis of inputimage data and the information of the object that is preferable for thesuperposition of the advertisement image data to be superposed, which isinput from the control section 340, and outputs the informationprescribing the area of the object to the preprocessing section 326 andthe image superposing section 327.

The constitution and the operations of the image superposing areaextracting section 321 will be described in detail by reference to FIG.5.

FIG. 5 is a block diagram showing the constitution of the imagesuperposing area extracting section 321 shown in FIG. 4.

The image superposing area extracting section 321 has a featureextracting section 322, a knowledge database 323, a candidate objectdetecting section 324 and a superposing area deciding section 325.

The feature extracting section 322 extracts a prescribed feature points,for example, a changing point of luminance, a changing point of color, apoint of a specific color and the like, and outputs the information ofthe extracted feature points to the candidate object detecting section324.

The knowledge database 323 is a database storing the objects in inputimage data that the image superposition area extracting section 321 isgoing to recognize in a form prescribed by feature points beingdetection objects of the candidate object detecting section 324. Herein,the aforesaid objects in input image data are image data generated bytaking of the data of objects having a possibility of being replaced byadvertisement image data or of being superposed with the advertisementimage data by the user terminal apparatus 100. A plurality of data ofeach object are recorded at every state corresponding to some typicalimage data having high possibilities of being generated in cases wherethe user terminal apparatus 100 are ordinarily used to take ordinarilyconceivable scenes.

Moreover, the knowledge database 323 stores other features such as “theface and the hands have the same color as that of the skin.” besides thedispositions of the feature points of each object, or features ofrelative positional relationships of each object, for example, “anouterwear is disposed below a face, and hands locate on the both sidesthereof”

The candidate object detecting section 324 compares the configuration ofa feature point extracted by the feature extracting section 322 with thedata of each object recorded in the knowledge database 323, and refersthe other features other than the disposal of the feature point and thefeatures of relative positional relationships with the already detectedobjects, and thereby detects the area of a specific object having apossibility of being replaced or superposed by advertisement image datain the input image data. The information of the detected object isoutput to the superposing area deciding section 325.

The superposing area deciding section 325 decides an object forsuperposing a final advertisement image data on the basis of theinformation of the detection result of the object input from thecandidate object detecting section 324 and the information that ispreferable for the superposing of the advertisement image data and isinput from the control section 340. The information of the objects thatare possible to be superposed and are beforehand set at every sellinggoods and the information of the order of priority of the objects areinput into the superposing area deciding section 325 from the controlsection 340. Accordingly, the superposing area deciding section 325checks whether or not any object possible to be superposed is detectedby the candidate object detecting section 324 in sequence from thehigher order of priority, and decides the object that is detected as theobject being the superposed object.

Then, the superposing area deciding section 325 outputs the informationthat prescribes the area of the object from the image superposed areaextracting section 321.

The aforesaid is the description of the constitution of the imagesuperposing area extracting section 321.

The preprocessing section 326 of the advertisement information addingsection 320 converts the advertisement image data of the superposedobject read out from the advertisement information database 380 andinput through the database I/F 330 to data having a size and a shapepossible to be superposed to the area of the superposing objectextracted by the image superposing area extracting section 321, andoutputs the converted data to the image superposing section 327.

The image superposing section 327 superposes the data in the superposingobject area input from the image superposing area extracting section 321and the advertisement image data input, from the preprocessing section326 among the input data, and outputs the image data of the result ofthe superposing to the switching section 328. In the present embodiment,the image superposing section 327 merely replaces the data in the areaof the superposing object of the input image data with the advertisementimage data and outputs the replaced data.

The switching section 328 selects input image data in the case where anuser did not set to add advertisement information, and selects the imagedata superposed by the image superposing section 327 in the case wherethe user set to add the advertisement information on the basis of thecontrol signal from the control section 340, and then the selected dataare respectively output to the communication I/F 310.

The aforesaid is the description of the constitutional sectionconcerning the processing of the image data in the advertisementinformation adding section 320.

The database I/F 330 of the service business enterprise system 300 readsout the advertisement information of desired selling goods from theadvertisement information database 380 under the control of the controlsection 340, and outputs the read out advertisement information to theadvertisement information adding section 320 through a prescribedchannel. A plurality of input-output channels possible to output imagedata concurrently are provided between the database I/F and theadvertisement information adding section 320, and the database I/F 330virtually concurrently reads out a plurality of pieces of advertisementinformation and outputs the read out advertisement information to theadvertisement information adding section 320 through the prescribedchannels.

Incidentally, the advertisement information read out from theadvertisement information database 380 has the advertisement image dataand the additional information of the selling goods.

The user database 350 is a kind of database which records the variousmanagement data of the user who uses the communication service providedby the service business enterprise system 300.

The user database 350 records the management data such as the name, theID (Identification), the destination information of connectionmanagement data of each user and the adjustment data concerning theutilization of services. Moreover, the setting whether each userapproves the usage of his or her live image as an advertising medium ornot is also recorded.

The sponsor database 360 is a kind of database storing various data onthe sides of sponsors which use the advertisement propagandizingservices to the live image in the private communication by the servicebusiness enterprise system 300.

The sponsor database 360 stores the information such as the sponsor ofeach selling goods that are desired to be advertised, the destinationinformation of connection to the sponsor, the address information forthe reading out of the advertisement information of the selling goods bythe advertisement information database 380, and a preferable superposedform at the time of the superposing of the selling goods to the liveimage signal.

The contract-related database 370 stores the information such as thecontracts with the users, who approved the usage of their live imagesignals as the advertisement media, for the actual usage as theadvertisement media, the setting thereof, and the conditions thereof. Toput it concretely, information such as the selling goods, the ways ofexchanging the advertisements, the advertisement orders of priority, andthe consideration receiving ways of advertisement objects selected byeach user are recorded.

The advertisement information database 380 is a kind of databaserecording the advertisement information of each selling goods of theadvertisement object is recorded.

In the advertisement information database 380, the advertisementinformation of every selling goods, i.e. the image data and theadditional information of the selling goods are recorded. The additionalinformation contains the connection information of the sponsor system400 of the sponsor of selling goods and the information that prescribesthe operations of the user terminal apparatus 100 when the image dataare chosen by the user.

Then, the information concerning the selling goods of an advertisement,object is read out at high speed by the database I/F 330 from theadvertisement information database 380. Moreover, the advertisementinformation of new selling goods transmitted from a sponsor isappropriately recorded in the advertisement information database 380through the database I/F 330.

The control section 340 appropriately adds advertisement information todata to be transmitted between users, and controls each section of theservice business enterprise system 300 so that the service businessenterprise system 300 can appropriately provide advertisement propagandaenvironment to the user terminal apparatus 100 and the sponsor system400 that are connected by the connection system 200.

To put it concretely, first, the control section 340 performs desiredprocessing correspondingly to various demands which are input from theuser terminal apparatus 100 and the sponsor system 400 through thecommunication I/F 310.

For example, in the case where the sponsor system 400 makes demands suchas the utilization of the advertisement service provided by the servicebusiness enterprise system 300, and the new registration and the erasureof selling goods of an advertisement object, the control section 340appropriately updates the sponsor database 360 and the advertisementinformation database 380 on the basis of the demands.

Moreover, in the case where a new user demands his or her registration,the control section 340 appropriately performs sessions with the userterminal apparatus 100 through the communication I/F 310 to obtainvarious pieces of information which are necessary to the registration,and the control section 340 performs the investigation of the user asthe occasion demands. Then, if the user passes the investigation, thecontrol section 340 records the information in the user database 350 andregisters the user.

Moreover, in the case where the user terminal apparatus 100 performs thesetting as to whether the user approves that a live image of the user tobe transmitted to a communication companion is utilized as anadvertisement medium or not, and in the case where the user selectsselling goods or demands to change the setting or the selection in thecase where the user approves the utilization, the control section 340updates the contents of the contract-related database 370 on the basisof the user's demands.

Moreover, when the user terminal apparatus 100 demands the communicationwith another user terminal apparatus 100, the control section 340instructs the communication I/F 310 to connect the demanded userterminal apparatus 100 with the user terminal apparatus 100 of thecommunication companion.

Then, referring to the user database 350, the control section 340 judgeswhether or not the user terminal apparatus 100 that demanded thecommunication and the user terminal apparatus 100 of the communicationcompanion are respectively performed the setting to use their liveimages as advertising media.

When the user terminal apparatus 100 performed such setting, referringto the contract-related database 370, the control section 340 reads outthe contents of the setting, i.e. the information which selling goodsare changed at which tuning while performing the superposing of theadvertisement information of the selling goods.

Moreover, the control section 340 instructs the database I/F 330 to readout the advertisement information of the selling goods to be superposedand to output the read out advertisement information to a prescribedchannel of the advertisement information adding section 320.

Moreover, the control section 340 controls the operations of theadvertisement information adding section 320 so that advertisement imagesignals are appropriately superposed according to the setting read outfrom the contract-related database 370 when communication is begunbetween the user terminal apparatus 100.

Next, the operations of such communication system 11 will be describedby reference to FIGS. 6-8.

FIG. 6 is a figure for illustrating the operations of the communicationsystem 11.

First, an user A performs the setting of the use of the user's liveimage signal as the advertising medium beforehand and the selection ofcommodities to the service business enterprise system 300.

When the user A tried to communicate with an user B by the visualtelephone-form using live image signals under such a situation, the userA demands the connection with the user terminal apparatus 100-B of theuser B from the user terminal apparatus 100-A to the service businessenterprise system 300 through the connection system 200-(1).

In the service business enterprise system 300, the demand is input intothe control section 340 through the communication I/F 310, and thedemand of the connection to the user terminal apparatus 100-B of theuser B is made from the communication I/F 310 to the connection system200 under the control of the control section 340.

Each circuit connection among the user terminal apparatus 100-A of theuser A, the user terminal apparatus 100-B of the user B and the servicebusiness enterprise system 300 is established by the calling of the userterminal apparatus 100-B of the user B by the connection system 200 andby the setting of the user terminal apparatus 100-B by the user B to atelephone calling state (200-(2)).

On the other hand, the control section 340 of the service businessenterprise system 300 retrieves the information of the user A and theuser B in the user database 350 when the control section 340 receivesthe connection demand from the user A, and checks whether the user A andthe user B respectively performed the setting to use each live imagesignal as an advertisement medium or not.

As a result, because the user A performed such setting, the controlsection 340 reads out the information such as the selling goods,changing conditions of the selling goods, and consideration receptionmethods from the contract-related database 370, and the control section340 reads out the information such as the address of the advertisementinformation database 380 in which the advertisement information of theselling goods is recorded and the superposed form at the time of thesuperposing of the selling goods from the sponsor database 360.

Then, on the basis of the read out information, the control section 340instructs the database I/F 330 to read out the data of the selling goodsof the advertisement object from the advertisement information database380, and instructs the advertisement information adding section 320 tosuperpose the image data of the selling goods to a live image signal ina fashion that the user and the sponsor desire, respectively.

When the live image signal to be transmitted to the user terminalapparatus 100-B is input to the service business enterprise system 300from the user terminal apparatus 100-A of the user A in such a state,the advertisement information adding section 320 superposes the imagesignal of the selling goods of the advertisement, object on the liveimage signal, and the superposed image signal is transmitted to the userterminal apparatus 100-B of the user B.

For example, in the case where a live image signal 500 shown in FIG. 7is input, and the selling goods of the advertisement object are ahamburger, and further the superposed form of the selling goods is acondition of “printing the picture of the selling goods on the clothes”,an image that a pattern 502 of a hamburger is printed on the T shirtthat the user A wears as shown in FIG. 8 is transmitted to the user B.

Moreover, at this time, the information such as the area where the imagedata of the advertisement object as selling goods is displayed, theaddress of the sponsor system 400 of the sponsor of the selling goods,and the destination of connection is transmitted to the user terminalapparatus 100-B of the user B at the same time as control data togetherwith the superposed image data.

The processing in the advertisement information adding section 320 forgenerating a superposing image signal as shown in FIG. 8 will bedescribed.

First, feature points such as the luminance changing points, the colorchanging points and the representative points of colors of the liveimage signal as shown in FIG. 7 are detected by the feature extractingsection 322 in the image superposing area extracting section 321 of theadvertisement information adding section 320.

Next, the information stored in the knowledge database 323 where theshapes of faces, hands and outerwear are beforehand recorded is activelyutilized by the candidate object detecting section 324, and objects suchas those faces, hands and outerwear are detected in the image signalwhich is composed of the feature points. For example, objects such asthe face, the hands and the outerwear are detected in the image as shownin FIG. 7.

Then, the object according to the information of the e superposed formof the advertisement object as selling goods input from the controlsection 340 is selected among the detected objects by the superposingarea deciding section 325, and the area information of the selectedobject is output to the preprocessing section 326 and the imagesuperposing section 327. For example, if superposed forms such as “onthe plane in the background”, “outerwear” and “hand” are set to theselling goods of “hamburger” in the order, the “outerwear” at the secondplace in the order agrees with the information. Consequently, the areainformation in the live image of the outerwear is output.

Next, the preprocessing section 326 in the advertisement informationadding section 320 transforms the image data input from theadvertisement information database 380 through the database I/F 330,i.e. the image data of “hamburger” to the image data having anappropriate size and shape for the embedding into the area detected bythe image superposing area extracting section 321, and outputs theresulted image data to the image superposing section 327.

Then, in the image superposing section 327, the image data of thehamburger input from the preprocessing section 326 is superposed in anarea of the outerwear of the live image signal input from the imagesuperposing area extracting section 321, and then the image as shown inFIG. 8 is generated.

The generated superposed image data is selected by the switching section328 and is transmitted to the user terminal apparatus 100-B of the userB through the communication I/F 310.

Now, when the user B who is talking with the user A over the telephoneand to whom an image, for example, shown in FIG. 8 is transmitted takesan interest in the hamburger printed on the T shirt of the user A, it isexpected that the user B asks the user A what the hamburger is as anordinary action.

On the other hand, the possibility that the user A has some knowledgeabout the hamburger and has a good feeling to the hamburger is highbecause they are the selling goods that the user A selected as anadvertisement object of his or her own accord.

Consequently, if the user B asks the user A about the hamburger, thereis high possibility that the user A explains the hamburger being theselling goods and gives utterance recommending and urging the purchaseof the selling goods by the remarks such as “it is delicious” or “whynot try it once”.

As a result, the effect of the advertisement propaganda is obtained morethan the image of the object is simply provided to the user B.

Regardless of the existence of such a conversation between the users,when the user B takes an interest in the hamburger and wants to know thedetailed information, for example, the price thereof, the selling periodthereof, the nearest purchasable place and the like, the user B performsthe selection operation of the hamburger on the user terminal apparatus100-B.

As mentioned above, the information indicating the image area of theadvertisement object as selling goods, the destination information ofconnection of the sponsor system 400 of the sponsor of the selling goodsand the information prescribing the operations of the user terminalapparatus 100-B in the case where the image of the selling goods isselected as the need arises are transmitted to the user terminalapparatus 100-B in addition to the live image including the superposingimage signal of the advertisement object as selling goods.

Consequently, the user terminal apparatus 100-B performs the prescribedoperation by the selection of the image of the selling goods by the userB. In the present embodiment, by the selection of the image data of theselling goods, the demand of information or the demand of communicationis made by the user terminal apparatus 100-B to the sponsor system 400of the sponsor of the selling goods.

As a result, the sponsor system 400 accesses to the user terminalapparatus 100-B of the user B, and then the further information of theselling goods is transmitted. Moreover, in some cases, the purchase ofthe selling goods can be applied, too, (200-(3)).

Incidentally, along with such processing, the control section 340 of theservice business enterprise system 300 totals up the information of theactual results of the processing concerning the advertisementpropaganda, namely the number of times or the period of time of theusage after the superposed of the image data of the selling goods, thenumber of times of the selection of the selling goods at the destinationof advertisement, the fact whether the selling goods are finallypurchased or not, and the like, at every user who is advertisementmedium, and at every selling goods, and stores the totaled upinformation in the user database 350 and the sponsor database 360.

Then, the control section 340 performs the processing of paying aprescribed consideration to each user who is an advertising medium onthe basis of the information of the actual usage results appropriately.

As described above, according to the communication system 11 of thepresent embodiment, a live image which is used in a privatecommunication environment among ordinary users can be used as anadvertisement propaganda environment of the selling goods or anelectronic transaction environment.

Then, especially, such advertisement propaganda is performed in a waythat an user advertises the selling goods which the user selectedpersonally on the basis of the experience and the judge of himself orherself to the private companion. Consequently, the effect thereof isvery high.

Incidentally, the present invention, is not limited to the presentembodiment and arbitrary and suitable various alterations are possible.

For example, the superposing form of the image signal of selling goodsperformed by the advertisement information adding section 320 of theservice business enterprise system 300 is not limited to the exampleshown in FIG. 8, and a various forms are available.

For example, the place where the image data of selling goods is embeddedmay be an optional place in a transmitted live image.

For example, an image 503 of selling goods may be superposed in ahorizontal plane part in the background image as shown in FIG. 9, and animage 504 of selling goods may be superposed in a perpendicular planepart in the background image as shown in FIG. 10. In the former case,the image with reality as if the selling goods are placed on thehorizontal plane such as a desk can be obtained, and in the latter case,the image with reality as if the selling goods are hung on a wall or apicture of the selling goods is hung on the wall can be obtained.

Moreover, in the first place, the image of a selling goods may besuperposed in a form as if the user taken by a video camera in the liveimage is using the selling goods.

For example, there may be produced an image in which the outerwear ofthe user is replaced by closes 505 of an advertisement object in thefirst place as shown in FIG. 11 and an image in which the user hasselling goods 506 by hand as shown in FIG. 12.

According to such a way, such image with more reality can be generated.

Moreover, as shown in FIG. 13, the advertisement and propaganda ofselling goods may be performed in a way in which an object 507 havingthe name of the selling goods or the image signal of the selling goodsis simply superimposed on a live image signal regardless of the liveimage.

Such a method has small reality, but the structure of an imageprocessing circuit can be simplified very much.

Such a basic way, too, is within the scope of the present invention.

Moreover, although it is not shown in the aforesaid embodiment, in thecase where the user B who is talking with the user A over the telephonetook an interest in advertised selling goods and begin to communicatewith the sponsor system 400 concerning the selling goods, the furtherinformation of the selling goods transmitted from the sponsor system 400is simultaneously displayed on the display section 108 of the userterminal apparatus 100-B on which a live image of the user A is beingdisplayed.

The forms of the display in this case include the cases shown in FIGS.14A-14D, for example, and may be arbitrary forms.

For example, as shown in FIG. 14A, a window in which the live image ofthe user A is displayed and a window in which an introduction image ofselling goods is displayed may be displayed in a way that both of thewindows are displayed with a shift while being overlapped partially, orboth the windows may be displayed separately in the vertical directionas shown in FIG. 14B, or both may be displayed separately in thehorizontal direction as shown in FIG. 14C.

Moreover, while the information of selling goods is transmitted from thesponsor system 400, even if the user B is talking with the user A overthe telephone, only the introduction image of the selling goods may bedisplayed on the whole screen of the display without the display of thelive image of the user A as shown in FIG. 14D.

Moreover, although the case where the information of selling goods isobtained from the sponsor system 400 while the user B keeps theconversation with the user A is described in the aforesaid embodiment,the information concerning the advertisement object as selling goodsthat are received during the telephone call with the user A is stored inthe memory 120 of the user terminal apparatus 100 until the nexttelephone call begins. Therefore, the information stored in the memory120 may be read out after the end of the telephone call with the user A,and the desired information may be obtained by the separate connectionwith the sponsor system 400.

Moreover, in the present embodiment, the information of an advertisementobject as selling goods is superposed only with image data, and audiodata are transmitted between users without any processing. However, inthe present invention, the advertisement information may be added toaudio data.

To put it concretely, for example, as the voices in the background whiletelephone calling is being practicing, music or voices making the userremember the selling goods or the company manufacturing or selling theadvertisement object.

Moreover, the audio data concerning an advertisement object as sellinggoods may be sounded when voiceless time has continued longer than apredetermined period, namely silence continues longer than thepredetermined period.

The constitution of a processing section for the superposing of voicesthat is provided in the advertisement information adding section 320 forthe aforesaid cases is shown in FIG. 15.

In the configuration shown in FIG. 15, a voice superposing partdetecting section 621 detects parts at which voices are superposed, anda preprocessing section 626 converts, for example, the levels, theperiods and the like of the audio data to be superposed into formssuitable to the superposition. Furthermore, an image superposing section627 performs the superposing, and a switching section 628 selects thedata to be output.

Moreover, although in the aforesaid service business enterprise system300 the advertisement object commodities are selected to be used inconformity with the priority set by each user, they may be selectedaccording to telephone calling companions to be objects of advertisementpropaganda.

For example, each user may select selling goods by reference to thepreviously registered information concerning the decision of sellinggoods when the user receives advertisements.

Moreover, as shown in FIG. 16, a hobby and preference database 390 inwhich the hobby and the preference of each user are registered isbeforehand provided in a service business enterprise system 300 b, andthen selling goods that seem to be effective at the time of beingadvertised to an user may be selected on the basis of the data stored inthe hobby and preference database 390. In this case, the data to beregistered in the hobby and preference database 390 may be collected onthe basis of the declaration from each user, and they may beautomatically collected by the extraction using key words from thecontents of conversations of the users, or the like.

Moreover, the constitutions of the connection system 200 and the servicebusiness enterprise system 300 in the communication system 11 are alsonot limited to those of the present embodiment.

For example, as shown in FIG. 17, the configuration of the communicationsystem may be a form in which a connection business enterprise alsosupplies services as a service business enterprise, namely it may be thecommunication system 12 in which the connection system 200 and theservice business enterprise system 300 are integrated to form aconnection business enterprise system 700.

Moreover, although the aforesaid embodiment includes a basicconfiguration in which one connection system 200 and one servicebusiness enterprise system 300 respectively provided, a configuration inwhich a plurality of connection systems 200 and a plurality of servicebusiness enterprise systems 300 are provided may be applicable.

Moreover, although the superposing of a live image and an advertisementimage is simply performed by replacing a prescribed area of the liveimage signal with the advertisement image signal in the aforesaidembodiment, the superposing is not limited to such processing. Forexample, the superposing may be performed by processing the blend ofinput image data and advertisement image data at a prescribed ratio. Inthe case where such a superposing is performed, an image like aperspective of an input image as a background image can be generated,e.g. semitransparent selling goods can be more realistically expressed.

Moreover, the data form when such a live image signal and a superposedimage signal are transmitted may be a personal format, or may be theformat of a HTML (Hyper Text Markup Language) or the i-MODE. Moreover,it may be an arbitrary opened format to be connectable to the Internet.

Moreover, the superposing of a live image signal and selling goods imagemay be performed by the user terminal apparatus 100 on the side oftransmission or the user terminal apparatus 100 on the side ofreception.

Moreover, on the occasion of the switchover to the mode of theelectronic transaction in which the information of selling goodsactually accessed from the advertisement, the access is not performeddirectly to the sponsor system 400 from the user terminal apparatus 100,but the service business enterprise system 300 may make the userterminal apparatus 100 and the sponsor system 400 connect by mediating.In such a form, the control data to be transmitted to the connectionsystem 200 from the service business enterprise system 300 in theadvertisement step can be decreased.

Moreover, an object of advertisement and propaganda may be an arbitraryone such as selling goods, a corporation and a brand.

Moreover, as for the timing at which an advertisement is changed, theadvertisement of one object may be changed at every telephone call, ormay be changed at every passage of a prescribed time. Furthermore, theadvertisement may be changed in conformity with the operation of an userwho transmits communication or an user who receives the communication.

Moreover, an arbitrary standard may be set as the payment standard of aconsideration that each user becomes an advertising medium. For example,the number of times of the appearance of an advertisement, the period oftime of the appearance of the advertisement, the selection number oftimes by an user on the side of reception, and every sales of theselling goods may be the standard.

Moreover, as to the payment method of the consideration in the case,too, an arbitrary method may be adopted. For example, the payment incash, returning as a utility rate of the communication service,returning at the time of performing electronic transactions and the likemay be applicable.

Moreover, although the communication between two users is illustrated inthe present embodiment, in case of communication among numerous users ofequal to or more than three, too, it is possible to apply the presentembodiment just as it is.

Besides, the way of the connecting of a connection business enterprise,the kind of the communication network, the transmission form of data,the form of the user terminal apparatus and so on, too, may be anarbitrary way, an arbitrary kind, arbitrary forms, respectively, andthey are not limited to the present embodiment at all.

Moreover, in the present embodiment, the information of selling goodswhich are once memorized in the memory 120 of the user terminalapparatus 100 is supposed to be erased at every time when the nextcommunication is begun by the user terminal apparatus 100. However, theinformation may be accumulated in sequence, and may be made to be ableto be referred as the occasion demands by the tracing back to the past.The information of selling goods may be utilized in an arbitrary formafter reception.

Although the invention has been described in its preferred form with acertain degree of particularity, obviously many changes and variationsare possible therein. It is therefore to be understood that the presentinvention may be practiced than as specifically described herein withoutdeparting from scope and the sprit thereof.

1-34. (canceled)
 35. A communication server comprising: a communicatorconfigured to receive a user ID of a user, video data from a terminalused by the user, and a setting of whether image data is used with thevideo data, and a store configured to store the setting; and acontroller configured to control the communicator to transmit to one ormore other terminals the video data and the image data when the settingindicates that the image data is used.
 36. The communication serveraccording to claim 35, wherein the communicator receives the video datain real time from the terminal.
 37. The communication server accordingto claim 35, wherein the image data is superposed onto the video datawhen the other terminals display both the video data and the image data.38. The communication server according to claim 37, wherein the imagedata is superposed onto the video data with a certain transparency rate.39. The communication server according to claim 37, wherein the imagedata is superposed onto a predetermined area of a display area of thevideo data.
 40. The communication server according to claim 37, whereinthe controller detects feature points of an object within the videodata, and wherein the image data is superposed onto the video data basedon the feature points of the object.
 41. The communication serveraccording to claim 35, wherein the communicator receives a request foradditional information in response to a designation to the image databeing displayed on the other terminals.
 42. The communication serveraccording to claim 35, wherein the controller determines a total numberof usages of the image data in a predetermined period of time.
 43. Thecommunication server according to claim 35, wherein communication withthe terminals is done by HTML.
 44. A terminal comprising: a camera; acommunicator configured to transmit to a server a user ID, video datacaptured by the camera, and a setting of whether image data is used withthe video data; and a display configured to display video data withimage data.
 45. The terminal according to claim 44, wherein thecommunicator transmits the video data to the server in real time. 46.The terminal according to claim 44, wherein the display displays theimage data being superposed onto the video data.
 47. The terminalaccording to claim 46, wherein the display displays the image data witha certain transparency rate.
 48. The terminal according to claim 46,wherein the display displays the image data on a predetermined area of adisplay area of the video data.
 49. The terminal according to claim 46,wherein the display displays the image data on a display area of thevideo data based on detected feature points of an object within thevideo data.
 50. The terminal according to claim 44, further comprising apointing device, and wherein the communicator transmits a request foradditional information in response to a designation by the pointingdevice to the image data being displayed on the other terminals.
 51. Theterminal according to claim 50, wherein the display displays theadditional information so that the additional information is partiallyoverlapped with the video data.
 52. The terminal according to claim 50,wherein the display displays the additional information in a displayarea divided from a display area of video data.
 53. The terminalaccording to claim 50, wherein the display makes the video datadisappear and displays the additional information.
 54. The terminalaccording to claim 50, further comprising an authentication device, andwherein the communicator transmits a purchase request of a productrelated to the additional information, and wherein the authenticationdevice authenticates the purchase request.
 55. The terminal according toclaim 44, wherein communication with the server is done by HTML.
 56. Theterminal according to claim 44, wherein the terminal is a handhelddevice.
 57. A method of communicating with a terminal, comprising:receiving a user ID of a user, video data from a terminal used by theuser, and a setting of whether image data is used with the video data,storing the setting; and transmitting to one or more other terminals thevideo data and the image data when the setting indicates that the imagedata is used.
 58. The method of claim 57, wherein the video data isreceived in real time from the terminal.
 59. The method of claim 57,wherein the image data is superposed onto the video data when the otherterminals display both the video data and the image data.
 60. The methodof claim 59, wherein the image data is superposed onto the video datawith a certain transparency rate.
 61. The method of claim 59, whereinthe image data is superposed onto a predetermined area of a display areaof the video data.
 62. The method of claim 59, further comprising:detecting feature points of an object within the video data, and whereinthe image data is superposed onto the video data based on the featurepoints of the object.
 63. The method of claim 57, further comprisingreceiving a request for additional information in response to adesignation to the image data being displayed on the other terminals.64. The method of claim 57, further comprising determining a totalnumber of usages of the image data in a predetermined period of time.65. The method of claim 57, wherein communication with the terminals isdone by HTML.
 66. A method of communicating to a server, comprising:transmitting to a server a user ID, video data captured by a camera, anda setting of whether image data is used with the video data; anddisplaying video data with image data.
 67. The method of claim 66,wherein the video data is transmitted to the server in real time. 68.The method of claim 66 wherein the image data is displayed superposedonto the video data.
 69. The method of claim 68, further comprisingdisplaying the image data with a certain transparency rate.
 70. Themethod of claim 68, further comprising displaying the image data on apredetermined area of a display area of the video data.
 71. The methodof claim 68, further comprising displaying the image data on a displayarea of the video data based on detected feature points of an objectwithin the video data.
 72. The method of claim 66, further comprising:generating a designation by a pointing device to the image data beingdisplayed on the other terminals; and transmitting a request foradditional information when the image data is designated.
 73. The methodof claim 72, further comprising displaying the additional information sothat the additional information is partially overlapped with the videodata.
 74. The method of claim 72, further comprising displaying theadditional information in a display area divided from a display area ofvideo data.
 75. The method of claim 72, further comprising making thevideo data disappear and displaying the additional information.
 76. Themethod of claim 72, further comprising transmitting a purchase requestof a product related to the additional information, and authenticatingthe purchase request.
 77. The method of claim 66, wherein communicationwith the server is done by HTML.
 78. A non-transitory computer readablemedium on which is recorded a program for instructing a processor tocommunicate with a terminal, by steps comprising: receiving a user ID ofa user, video data from a terminal used by the user, and a setting ofwhether image data is used with the video data, storing the setting; andtransmitting to one or more other terminals the video data and the imagedata when the setting indicates that the image data is used.
 79. Themedium of claim 78, wherein the video data is received in real time fromthe terminal.
 80. The medium of claim 78, wherein the image data issuperposed onto the video data when the other terminals display both thevideo data and the image data.
 81. The medium of claim 80, wherein theimage data is superposed onto the video data with a certain transparencyrate.
 82. The medium of claim 80, wherein the image data is superposedonto a predetermined area of a display area of the video data.
 83. Themedium of claim 80, wherein the steps further comprise: detectingfeature points of an object within the video data, and wherein the imagedata is superposed onto the video data based on the feature points ofthe object.
 84. The medium of claim 78, wherein the steps furthercomprise receiving a request for additional information in response to adesignation to the image data being displayed on the other terminals.85. The medium of claim 78, wherein the steps further comprisedetermining a total number of usages of the image data in apredetermined period of time.
 86. The medium of claim 78, whereincommunication with the terminals is done by HTML.
 87. A non-transitorycomputer-readable medium on which is recorded a program for instructinga processor to communicate with a server, by steps comprising;transmitting to the server a user ID, video data captured by a camera,and a setting of whether image data is used with the video data; anddisplaying video data with image data.
 88. The medium of claim 87,wherein the video data is transmitted to the server in real time. 89.The medium of claim 87 wherein the image data is displayed superposedonto the video data.
 90. The medium of claim 89, further comprisingdisplaying the image data with a certain transparency rate.
 91. Themedium of claim 89, further comprising displaying the image data on apredetermined area of a display area of the video data.
 92. The mediumof claim 89, further comprising displaying the image data on a displayarea of the video data based on detected feature points of an objectwithin the video data.
 93. The mwsium of claim 87, further comprising:generating a designation by a pointing device to the image data beingdisplayed on the other terminals; and transmitting a request foradditional information when the image data is designated.
 94. The mediumof claim 93, further comprising displaying the additional information sothat the additional information is partially overlapped with the videodata.
 95. The medium of claim 93, further comprising displaying theadditional information in a display area divided from a display area ofvideo data.
 96. The medium of claim 93, further comprising making thevideo data disappear and displaying the additional information.
 97. Themedium of claim 93, further comprising transmitting a purchase requestof a product related to the additional information, and authenticatingthe purchase request.
 96. The medium of claim 87, wherein communicationwith the server is done by HTML.